Modality-Fused Graph Network for Cross-Modal Retrieval
نویسندگان
چکیده
Cross-modal hashing technology has attracted much attention for its favorable retrieval performance and low storage cost. However, existing cross-modal methods, the heterogeneity of data across modalities is still a challenge how to fully explore utilize intra-modality features not been well studied. In this paper, we propose novel approach called Modality-fused Graph Network (MFGN). The network architecture consists text channel an image that are used learn modality-specific features, modality fusion uses graph modality-shared representations reduce modalities. addition, integration module introduced channels features. Experiments on two widely datasets show our achieves better results than state-of-the-art methods.
منابع مشابه
MHTN: Modal-adversarial Hybrid Transfer Network for Cross-modal Retrieval
Cross-modal retrieval has drawn wide interest for retrieval across different modalities of data (such as text, image, video, audio and 3D model). However, existing methods based on deep neural network (DNN) often face the challenge of insufficient cross-modal training data, which limits the training effectiveness and easily leads to overfitting. Transfer learning is usually adopted for relievin...
متن کاملModeling Text with Graph Convolutional Network for Cross-Modal Information Retrieval
Cross-modal information retrieval aims to find heterogeneous data of various modalities from a given query of one modality. The main challenge is to map different modalities into a common semantic space, in which distance between concepts in different modalities can be well modeled. For crossmodal information retrieval between images and texts, existing work mostly uses off-the-shelf Convolutio...
متن کاملModality-specific Cross-modal Similarity Measurement with Recurrent Attention Network
Nowadays, cross-modal retrieval plays an indispensable role to flexibly find information across different modalities of data. Effectively measuring the similarity between different modalities of data is the key of cross-modal retrieval. Different modalities such as image and text have imbalanced and complementary relationships, which contain unequal amount of information when describing the sam...
متن کاملCorrelation Hashing Network for Efficient Cross-Modal Retrieval
Due to the storage and retrieval efficiency, hashing has been widely deployed to approximate nearest neighbor search for large-scale multimedia retrieval. Cross-modal hashing, which improves the quality of hash coding by exploiting the semantic correlation across different modalities, has received increasing attention recently. For most existing cross-modal hashing methods, an object is first r...
متن کاملCross-Modal Manifold Learning for Cross-modal Retrieval
This paper presents a new scalable algorithm for cross-modal similarity preserving retrieval in a learnt manifold space. Unlike existing approaches that compromise between preserving global and local geometries, the proposed technique respects both simultaneously during manifold alignment. The global topologies are maintained by recovering underlying mapping functions in the joint manifold spac...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEICE Transactions on Information and Systems
سال: 2023
ISSN: ['0916-8532', '1745-1361']
DOI: https://doi.org/10.1587/transinf.2022edl8069